Big Data, Data Science, and Civil Rights

Advances in data analytics bring with them civil rights implications. Data-driven and algorithmic decision making increasingly determine how businesses target advertisements to consumers, how police departments monitor individuals or groups, how banks decide who gets a loan and who does not, how employers hire, how colleges and universities make admissions and financial aid decisions, and much more. As data-driven decisions increasingly affect every corner of our lives, there is an urgent need to ensure they do not become instruments of discrimination, barriers to equality, threats to social justice, and sources of unfairness. In this paper, we argue for a concrete research agenda aimed at addressing these concerns, comprising five areas of emphasis: (i) Determining if models and modeling procedures exhibit objectionable bias; (ii) Building awareness of fairness into machine learning methods; (iii) Improving the transparency and control of dataand model-driven decision making; (iv) Looking beyond the algorithm(s) for sources of bias and unfairness—in the myriad human decisions made during the problem formulation and modeling process; and (v) Supporting the cross-disciplinary scholarship necessary to do all of that well. Over the past several years, government, academia, and the private sector have increasingly recognized that the use of big data and data science in more and more decisions has important implications for civil rights, from racial discrimination to income equality to social justice. We have seen many fruitful meetings and discussions, some of which are summarized briefly in an appendix below and have informed this report. However, a coherent research agenda for addressing these topics is only beginning to emerge. The need for such an agenda is critical and timely. Big data and data science have begun to profoundly affect decision making because the modern world is more broadly instrumented to gather data—from financial transactions, mobile phone calls, web and app interactions, emails, chats, Facebook posts, Tweets, cars, Fitbits, and on and on. Increasingly sophisticated algorithms can extract patterns from that data, enabling important advances in science, medicine, and commerce. As described in a recent 60 Minutes segment, for instance, IBM's Watson has helped doctors identify treatment strategies for cancer. Xerox now cedes hiring decisions for its 48,700 call-center jobs to software, cutting attrition by a fifth. And if you use the web, you have received advertisements targeted based on fine-grained details of your online behavior. Along with improved science and commerce come important civil rights implications. For example, data analytics tools can capture and instantiate decision-making patterns that are implicitly discriminatory— and can do so unintentionally, simply from distilling the data. Implicit discrimination by algorithms requires our attention because such data-driven methods are deployed in many of our most crucial social institutions. Risk assessment tools, for instance, are increasingly common in the criminal justice system, informing critical decisions like pre-trial detention, bond amounts, sentence lengths, and parole. Last year, ProPublica completed a study of a risk assessment tool employed in a number of courtrooms across that “Artificial Intelligence,” 60 Minutes, October 9, 2016, http://www.cbsnews.com/videos/artificial-intelligence/. 2 Walker, Joseph. “Meet the New Boss: Big Data: Companies Trade in Hunch-Based Hiring for Computer Modeling,” The Wall Street Journal, September 20, 2012, http://online.wsj.com/articles/SB10000872396390443890304578006252019616768.